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(54) Image retrieving apparatus performing retrieval based on coding information utilized for 
featured frame extraction or feature values of frames 



(57) An image retrieval information storing appara- 
tus capable of extracting a featured frame adaptive ly in 
the process of retrieval includes a coding information 
reading unit (101) reading prescribed coding informa- 
tion which will be information representing frame feature 
among coded image data, a frame feature value gener- 
ating unit (102) connected to the coding information 



reading unit (101) and generating a frame feature value 
which is a numerical representation of the frame feature 
based on the coding information, and a frame feature 
value storing unit (103) connected to the frame feature 
value generating unit ( 1 02) and storing the frame feature 
value in correspondence with each frame of the image 
data. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to an innage re- 
trieval information storing apparatus and an image re- 
trieving apparatus, retrieving an image based on the im- 
age retrieval information. More specifically, the present 
invention relates to an image retrieval information stor- 
ing apparatus and an image retrieving apparatus, using 
coding information utilized for featured frame extraction 
or feature values of respective frames, as image retriev- 
al information. 

Description of the Background Art 

[0002] When a user retrieves a motion picture, it is 
important that the user recognizes the contents of the 
motion picture, that is, what is the object picked up in 
the motion picture. Except when the motion picture has 
a text title or description, in most cases it is necessary 
for the user to confirm the contents by playing the whole 
motion picture. As the number of motion pictures among 
which a desired one is to be retrieved increases, it be- 
comes harder for the user to play the motion pictures 
one by one and to confirm the contents thereof, and the 
efficiency of retrieval is quite poor. The efficiency of re- 
trieval further lowers when each motion picture is in the 
form of coded compressed data requiring decoding and 
reproduction before retrieval. In order to solve this prob- 
lem, a method of retrieval may be effective in which one 
or a plurality of frames characteristic of the motion pic- 
ture, typically representing the contents of the motion 
picture are extracted and displayed, so that the contents 
of the motion picture can be displayed in schematic 
manner. 

[0003] The characteristic frame (hereinafter referred 
to as featured frame) used for motion picture retrieval 
includes, lor example, a scene change frame. Japanese 
Patent Laying-Open No. 9-261648 discloses a method 
in which a scene change frame is extracted, based on 
prediction mode information included in image data, 
from image data coded in accordance with interframe 
prediction coding. Over one frame, the number of I 
blocks which are coded in accordance with intratrame 
prediction, P blocks coded in accordance with forward 
interframe prediction, B blocks coded in accordance 
with backward interframe prediction and Bi blocks cod- 
ed in accordance with bidirectional interframe prediction 
are counted. Frequency information of respective 
blocks reflects correlation among frames in the motion 
picture. Thus, based on the frequency information of 
each block, a frame immediately following a scene 
change (= scene change frame) is detected. Further. 
Japanese Patent Laying-Open No. 10-23421 discloses 
a method of extracting a scene change frame, based on 



the change in amount of each piece of coding informa- 
tion contained in the coded image data. A conventional 
image retrieval information storing apparatus in accord- 
ance with the prior art is as shown in Fig. 14. 
5 [0004] Referring to Fig. 14, the conventional image re- 
trieval information storing apparatus includes a coding 
information reading unit 1401, a frame feature value 
generating unit 1402 connected to coding information 
reading unit 1401, an index information generating unit 
10 1403 connected to frame feature value generating unit 
1402, and an index information storing unit 1404 con- 
nected to index information generating unit 1403. 
[0005] Coding information reading unit 1401 reads, 
according to Japanese Patent Laying-Open No. 
IS 9-261648. block by block prediction mode information 
from coded image data, and reads, according to Japa- 
nese Patent Laying-Open No. 10-23421, respective 
pieces of coding information of different types. Frame 
feature value generating unit 1402 calculates, from the 
20 various pieces of coding information read by coding in- 
formation reading unit 1401, frequency of respective 
prediction mode blocks according to Japanese Patent 
Laying-Open No. 9-261648, and calculates accumulat- 
ed value of amount of respective pieces of coding infor- 
ms mation according to Japanese Patent Laying-Open No. 
10-23421. Index information generating unit 1403 de- 
termines the scene change frame which is the featured 
frame, based on the frame by frame feature value ob- 
tained in frame feature value generating unit 1402, that 
30 is, frequency information of respective prediction mode 
blocks according to Japanese Patent Laying-Open No. 
9-261648, and accumulated amount of respective piec- 
es of coding information according to Japanese Patent 
Laying-Open No. 10-23421. Index information generat- 
es ing unit 1 403 outputs positional information of the scene 
change frame as index information. Index information 
storing unit 1404 stores the index information output 
from the index information generating unit 1403 as im- 
age retrieval information. 
40 [0006] Fig. 15 represents a conventional image re- 
trieving apparatus using the image retrieval information 
stored in this manner. Referring to Fig. 15, the conven- 
tional image retrieving apparatus includes an image re- 
trieval executing unit 1501 executing retrieval using the 
45 image retrieval information and the coded image data 
as inputs and outputting the results of retrieval. 
[0007] The featured frame includes, in addition to the 
scene change frame, a scene change frame based on 
rapid/slow movement, disclosed in Japanese Patent 
so Laying-Open No. 9-284702. According to the invention 
disclosed in Japanese Patent Laying-Open No. 
9-284702, a motion vector for detecting a scene change 
frame is obtained from the original image, not from the 
coded information, and the scene change frame is de- 
55 tected based on the total sum of motion vectors in a 
frame. 

[0008] In the above described conventional image re- 
trieval information storing apparatus, positional informa- 
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tion of the extracted featured frame is obtained, and the 
positional information only is stored as the index infor- 
mation. Therefore, even when a- plurality of images 
which are the objects of retrieval have mutually different 
features, only a featured frame representing one same 
feature only can be obtained. If feature frames reflecting 
features of respective images are to be obtained, it is 
necessary tor the user to again generate the image re- 
trieval information from the coded image data. In the 
process of retrieval, when featured frames correspond- 
ing to various features of images which are the object 
of retrieval are to be taken out in an interactive manner, 
re-generation of the image retrieval information from the 
coded image data significantly lowers retrieval efficien- 
cy. Further, preparation of featured frame indexes cor- 
responding to every possible features for all the image 
data is not practical. Further, it is not possible by the 
prior art to meet with high efficiency a request for ex- 
tracting a featured frame which has not been expected 
by the provider providing the image retrieval informa- 
tion. 

SUMMARY OF THE INVENTION 

[0009] The present invention was made to solve the 
above described problems, and its object is to provide 
an image retrieval information storing apparatus capa- 
ble of immediately meeting a request for switching fea- 
tured frames in the process of retrieval and capable of 
extracting a featured frame adaptively. 
[0010] Another object of the present invention is to 
provide an image retrieving apparatus capable of high 
speed image retrieval processing. 
[0011] A still further object of the present invention is 
to provide an image retrieving apparatus capable of high 
speed image retrieval processing and allowing reduc- 
tion in capacity used for storing data necessary for re- 
trieval. 

[0012] A still further object of the present invention is 
to provide an image retrieving apparatus which can flex- 
ibly meet a user's request for extracting a featured 
frame. 

[0013] A still further object of the present invention is 
to provide an image retrieving apparatus which can flex- 
ibly meet a users request for extracting a featured frame 
and which is capable of high speed retrieval processing. 
[0014] According to an aspect of the present inven- 
tion, the image retrieval information storing apparatus 
includes: a coding information reading unit reading pre- 
scribed coding information which will be information rep- 
resenting frame features among coded image data; a 
frame feature value generating unit connected to the 
coding information reading unit, generating a frame fea- 
ture value which is a numerical representation of frame 
features, based on the coding information; and a frame 
feature value storing unit connected to the frame feature 
value generating unit, and storing the frame feature val- 
ue in correspondence with each frame of the image da- 



ta. 

[0015] The frame feature value of each frame neces- 
sary for extracting a featured frame is stored. Therefore, 
in the process of retrieval, it is possible to meet a request 
5 for switching a featured frame immediately, whereby 
adaptive extraction of a featured frame is possible. 
[001 6] According to another aspect, the image retriev- 
al information storing apparatus includes: a coding in- 
formation reading unit reading prescribed coding infor- 
mation which will be information representing frame fea- 
tures among coded image data; and a coded informa- 
tion storing unit connected to the coding information 
reading unit, and storing coding information in corre- 
spondence with each frame of the image data. 
[0017] The coding information necessary for extract- 
ing a featured frame is stored. Therefore, in the process 
of retrieval, it is possible to meet a request for switching 
a featured frame immediately, whereby adaptive extrac- 
tion of a featured frame is possible. 
[001 8] According to a still further aspect, an image re- 
trieval information storing apparatus includes: a coding 
information reading unit reading prescribed coding in- 
formation which will be information representing frame 
features among coded image data; a frame feature val- 
ue generating unit connected to the coding information 
reading unit, and generating a frame feature value which 
is a numerical representation of frame features based 
on the coding information; and a storing unit connected 
to the coding information reading unit and the frame fea- 
ture value generating unit, and storing the coding infor- 
mation and the frame feature value in correspondence 
with each frame of the image data. 
[0019] The frame feature value frame by frame and 
the coding information necessary for extracting a fea- 
tured frame are stored. Therefore, in the process of re- 
trieval, it is possible to meet a request for switching a 
featured frame immediately, whereby adaptive extrac- 
tion of a featured frame is possible. 
[0020] In accordance with a further aspect of the 
present invention, the image retrieving apparatus in- 
cludes: an index information generating unit receiving a 
frame feature value which is a numerical representation 
of frame features corresponding to each frame of the 
coded image data, determining a featured frame among 
the image data based on the frame feature value in ac- 
cordance with a request for extracting a featured frame, 
and generating index information which is positional in- 
formation of the featured frame; and a image retrieval 
executing unit connected to the index information gen- 
erating unit, transmitting a request for extracting a fea- 
tured frame to the index information generating unit, re- 
ceiving the index information from the index information 
generating unit, receiving image date from outside and 
outputting a frame specified based on the index infor- 
mation. 

[0021] The featured frame is extracted based on the 
frame feature value. Therefore, high speed retrieval 
processing is possible, and capacity used for storing da- 
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ta necessary for retrieval can be reduced. 
[0022] In accordance with a still further aspect, the im- 
age retrieving apparatus includes: a frame feature value 
generating unit receiving prescribed coding information 
included in coded image data, which will be information 
representing frame features, and generating a frame 
feature value which is a numerical representation of the 
frame features, based on coding information in accord- 
ance with a request for extracting a frame feature value; 
an index information generating unit connected to the 
frame feature value generating unit, receiving the frame 
feature value from the frame feature value generating 
unit, determining a featured frame among the image da- 
ta based on the frame feature value in response to a 
request for extracting the featured frame, and generat- 
ing index information which is positional information of 
the featured frame; and a image retrieval executing unit 
connected to the frame feature. value generating unit 
and the index information generating unit, transmitting 
a request for extracting frame feature value to the frame 
feature value generating unit, transmitting a request for 
extracting a featured frame to the index information gen- 
erating unit, receiving index information from the index 
information generating unit, receiving image data from 
outside, and outputting a frame specified based on the 
index information. 

[0023] The frame feature value is generated based on 
the coding information. Therefore, when there is a new 
request for extracting a frame feature value, the method 
of generating the frame feature value for extracting the 
feature frame is changed in the process of retrieval. 
Therefore, it is possible to flexibly meet the user's re- 
quest for extracting the featured frame. 
[0024] In accordance with a still further aspect of the 
present invention, the image retrieving apparatus in- 
cludes: a frame feature value generating unit receiving 
prescribed coding information included in coded image 
data, which will be information representing frame fea- 
tures, and generating a frame feature value which is a 
numerical representation of frame features based on the 
coding information in response to a request for extract- 
ing the frame feature value; an index information gen- 
erating unit connected to the frame feature value gen- 
erating unit, receiving frame feature values from the 
frame feature value generating unit and from the out- 
side, determining a featured frame among image data 
based on the frame feature values in response to a re- 
quest for extracting a featured frame, and generating in- 
dex information which is positional information of the 
featured frame; and a image retrieval executing unit 
connected to the frame feature value generating unit 
and the index information generating unit, transmitting 
a request for extracting frame feature value to the frame 
feature value generating unit, transmitting a request for 
extracting a featured frame to index information gener- 
ating unit, receiving index information from the index in- 
formation generating unit, receiving image data from the 
outside, and outputting a frame specified based on the 



index information. 

[0025] A new featured frame is extracted in accord- 
ance with the request for extracting the featured frame 
and the request for extracting frame feature value gen- 
s erated in the process of executing image retrieval, and 
with the coding information and the frame feature value. 
Accordingly, flexible image retrieval is possible, and 
high speed retrieving process based on the frame fea- 
ture value is possible. 

[0026] The foregoing and other objects, features, as- 
pects and advantages of the present invention will be- 
come more apparent from the following detailed de- 
scription of the present invention when taken in conjunc- 
tion with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0027] Fig. 1 represents a configuration of an image 
retrieval Information storing apparatus in accordance 
with a first embodiment of the present invention. 
[0028] Fig. 2 represents internal configuration of a 
frame feature value generating unit. 
[0029] Fig. 3 represents reference relation of frames 
in prediction coding method. 

[0030] Fig. 4 represents a configuration of an image 
retrieving apparatus In accordance with the first embod- 
iment of the present invention. 

[0031] Fig. 5 represents an internal configuration of 
an index information generating unit. 
[0032] Fig. 6 represents a configuration of the image 
retrieval information storing apparatus in accordance 
with the first embodiment of the present invention. 
[0033] Fig. 7 represents a configuration of the image 
retrieving apparatus in accordance with the first embod- 
iment of the present invention. 

[0034] Fig. 8 represents a configuration of the image 
retrieval information storing apparatus in accordance 
with a second embodiment of the present invention. 
[0035] Fig. 9 represents a configuration of the image 
retrieving apparatus in accordance with the second em- 
bodiment of the present invention. 
[0036] Fig. 1 0 represents a configuration of the image 
retrieval Information storing apparatus in accordance 
with the second embodiment of the present invention. 
[0037] Fig. 1 1 represents a configuration of the image 
retrieving apparatus in accordance with the second em- 
bodiment of the present invention. 
[0038] Fig. 1 2 represents a configuration of the image 
retrieval Information storing apparatus in accordance 
with a third embodiment of the present invention. 
[0039] Fig. 1 3 represents a configuration of the image 
retrieving apparatus in accordance with a third embod- 
iment of the present invention. 

[0040] Fig. 1 4 represents a configuration of a conven- 
tional image retrieval information storing apparatus. 
[0041] Fig. 1 5 represents a configuration of a conven- 
tional image retrieving apparatus. 
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DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0042] In the following description of the embodi- 
ments of the present invention, a scene change frame 
and a frame in which rapid movement is locally observed 
are extracted as featured frames, by way of example, 
using motion vector information and coding prediction 
mode information among coding information included in 
the image data. The configuration of the present inven- 
tion is also applicable when other coding information is 
used or when other featured frames are to be extracted. 

First Embodiment 

[0043] Referring to Fig. 1 , the image retrieval informa- 
tion storing apparatus in accordance with the first em- 
bodiment includes a coding information reading unit 
101, a frame feature value generating unit 102 connect- 
ed to coding information reading unit 101, and a frame 
feature value storing unit 103 connected to frame fea- 
ture value generating unit 102. 

[0044] Coding information reading unit 101 receives 
coded image data as an input, and reads and outputs 
desired coding Information included in the image data. 
Coding information reading unit 101 in the present em- 
bodiment extracts and outputs prediction mode informa- 
tion and motion vector information from the coded image 
data. 

[0045] Frame feature value generating unit 1 02 proc- 
esses each piece of coding information obtained from 
coding information reading unit 101 , and generates and 
outputs a feature frame value for each frame as a refer- 
ence for determination of a featured frame. 
[0046] Frame feature value storing unit 1 03 stores, in 
a prescribed storing location, an output from frame fea- 
ture value generating unit 102. Details of the frame fea- 
ture value storing unit 103 will be described later. 
[0047] Referring to Fig. 2, frame feature value gener- 
ating unit 102 includes a prediction mode counting unit 
201 connected to coding information reading unit 101, 
frequency Information converting unit 202 connected to 
prediction mode counting unit 201 , a valid motion vector 
detecting unit 203 connected to coding information 
reading unit 101, a motion vector statistic calculating 
unit 204 connected to coding information reading unit 
101 and valid motion vector detecting unit 203, and a 
statistic information converting unit 205 connected to 
prediction mode counting unit 201 and motion vector 
statistic calculating unit 204. 

[0048] The prediction mode information read by cod- 
ing information reading unit 1 01 includes prediction cod- 
ing method for each frame, and prediction method for 
each coding block in the frame. Prediction mode count- 
ing unit 201 counts the number of blocks coded in ac- 
cordance with respective prediction methods, frame by 
frame. In a forward prediction coding frame (P frame), 
the number of I blocks coded in accordance with intra- 



frame prediction and the number of P blocks coded in 
accordance with tnterframe prediction are counted, re- 
spectively. In a bidirectional prediction coding frame 
(frame coded by bidirectional prediction) (B frame ), the 

s number of I blocks coded in accordance with intraf rame 
prediction, the number of P blocks coded in accordance 
with forward interframe prediction, the number of B 
blocks coded in accordance with backward interframe 
prediction and the number of Bi blocks coded in accord- 

10 ance with bidirectional interframe prediction are count- 
ed, respectively. 

[0049] Further, prediction mode counting unit 201 out- 
puts, as block prediction mode frequency information for 
each frame, the total number Pp of P blocks for the P 
frame, and the total number Bf ( = P + BI) of forward 
interframe prediction blocks (total number of blocks cod- 
ed in accordance with fonward Interframe prediction) 
and total number Bb ( = B + Bi) of backward interframe 
prediction blocks (total number of blocks coded in ac- 
20 cordance with backward interframe prediction) for the B 
frame. 

[0050] Frequency information converting unit 202 
converts these pieces of frequency information to at- 
tribute information for each frame (In the following, the 

25 attribute information related to each frame will be re- 
ferred to as the "frame feature value"). In the present 
embodiment, each frame feature value is set such that 
It represents correlation between the frame of interest 
and an Immediately preceding frame. 

30 [0051] Referring to Fig. 3, an example of the relation 
of reference among P and B frames at the time of Inter- 
frame prediction coding will be described. A starting 
point of an arrow represents a frame which is an object 
of coding/decoding, and the end point of the arrow rep- 

35 resents a reference frame at the time of coding/decod- 
ing. When the reference characters Pp, Bfn, and Bbn ( 
n = 1 ,2,3) allotted to respective arrows represent the 
aforementioned frequency information, the frame fea- 
ture value for frame B2, that is, frequency information 

40 related to correlation between frames BI and B2 should 
be calculated by a prescribed operation from respective 
block frequency Information Pp, Bfn. and Bbn. 
[0052] In the present embodiment, the value used as 
the frame feature value of frame B2 is the value of fre- 

4S quency information Bf2 when frequency information Bf2 
exceeds a certain threshold value, the value of frequen- 
cy information Bbl when frequency information Bf2 
does not exceed the certain threshold value but frequen- 
cy information Bbl exceeds the threshold value, and 

50 othenvise the value of frequency information Bf2 or Bbl 
which is larger. The frame feature value of frame B3 is 
calculated in the similar manner. As to frame BI , the val- 
ue of frequency information Bfl is used as the frame fea- 
ture value. As to frame P4, the value of frequency infor- 
ms mation Bb3 or Pp which is larger is used as the frame 
feature value. The method of calculating attribute infor- 
mation for each frame is not limited to this. Further, the 
frequency Information of the coded blocks counted by 
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the prediction mode counting unit 201 nnay be directly 
regarded as frame feature values. 
[0053] The valid motion vector detecting unit 203 de- 
tects valid motion vector only, from the motion vector 
information read by coding information reading unit 101 . 
When a camera is moving while being focused on an 
object, for example, the object and the background of 
the picked up image may have motion vectors very 
much different in magnitude or direction. In such a case, 
valid motion vector detecting unit 203 extracts only that 
motion vector of an area of interest, for example, the 
area of the object. 

[0054] Possible method of separating the object area 
from the background may include a method in which an 
average of the motion vectors of the entire frame is cal- 
culated and the motion vectors out of the average are 
successively extracted, and a method in which areas 
are divided into two by clustering. If the contents of the 
motion picture come to be more complicated as the 
number of the objects increases, for example, a method 
of valid motion vector detection in accordance with the 
complexity may be used. If the image is not of such a 
nature, the valid motion vector detection may be unnec- 
essary. 

[0055] Motion vector statistic calculating unit 204 cal- 
culates an average value of the magnitudes of motion 
vectors corresponding to P block for the P frame over 
the entire frame, based on the prediction mode informa- 
tion obtained by coding information reading unit 101. 
Motion vector statistic calculating unit 204 calculates an 
average value of magnitudes of motion vectors corre- 
sponding to all the blocks (Bf) coded in accordance with 
the forward interframe prediction and all the blocks (Bb) 
coded in accordance with the backward interframe pre- 
diction, for the B frame, over the entire frame. The sta- 
tistics calculated may not be limited to the average value 
of vector magnitudes, and different statistics including 
average vector may be calculated. Further, the motion 
vector is normalized considering an interval between an 
object frame and a reference frame at the time of calcu- 
lation. 

[0056] Statistic information converting unit 205 con- 
verts each of the above described motion vector statis- 
tics obtained by motion vector statistic calculating unit 
204 to a frame feature value provided for each frame. 
Similar to the frame feature value based on the predic- 
tion mode information, the frame feature value based 
on the motion vector information in the present embod- 
iment is set such that it indicates the motion information 
between a frame of interest and an immediately preced- 
ing frame. At this time, the coding block frequency infor- 
mation obtained by prediction mode calculating unit 201 
is referred to in calculating the motion vector frame fea- 
ture value. 

[0057] Referring to Fig. 3, statistic information con- 
verting unit 205 uses as the motion vector frame feature 
value to be allotted to frame B2, an average of magni- 
tudes of all motion vectors corresponding to frequency 



information Bf2 and frequency information Bbl , when 
values Bf2 and Bb1 of frequency information both ex- 
ceed a certain threshold value for frame B2. Statistic in- 
formation converting unit 205 uses as motion vector 

s frame feature value, an average of magnitudes of mo- 
tion vectors corresponding to frequency information Bf2, 
when the value of frequency information Bf2 only ex- 
ceeds the threshold value. Statistic information convert- 
ing unit 205 uses as motion vector frame feature value, 

10 an average of magnitudes of motion vectors corre- 
sponding to frequency information Bbl , when frequency 
information Bbl only exceeds the threshold value. Oth- 
erwise, statistic information converting unit 205 uses 0 
as the motion vector frame feature value. The motion 

IS vector frame feature value allotted to frame B3 is calcu- 
lated in the similar manner. 

[0058] For frame B1, statistic information converting 
unit 205 uses as motion vector frame feature value, an 
average of magnitudes of motion vector corresponding 
20 to frequency information Bf 1 when the value of frequen- 
cy information Bfl exceeds a certain threshold value, 
and otherwise 0. 

[0059] For frame P4, statistic information converting 
unit 205 uses as motion vector frame feature value to 

25 be allotted to frame P4, an average of magnitudes of all 
motion vectors normalized, corresponding to frequency 
information Bb3 and frequency information Pp, when 
values of frequency information Bb3 and Pp both ex- 
ceed a certain threshold value. When the value of fre- 

30 quency information Bb3 only exceeds the threshold val- 
ues, statistic information converting unit 205 uses as 
motion vector frame feature value, an average of mag- 
nitudes of the motion vectors corresponding to frequen- 
cy information Bb3. Otherwise, statistic information con- 

35 verting unit 205 uses 0 as the motion vector frame fea- 
ture value. 

[0060] The method of calculating the motion vector 
frame feature value for each frame is not limited to the 
above. The motion vector statistics corresponding to re- 

40 spective blocks described above calculated by motion 
vector statistic calculating unit 204 may be regarded as 
motion vector frame statistics frame by frame. 
[0061] The frame feature value on prediction mode 
and the frame feature value on motion vector obtained 
in this manner are output from frame feature value gen- 
erating unit 102 and stored in prescribed storing loca- 
tions by frame feature value storing unit 1 03. The storing 
location of the frame feature value may be a location 
corresponding to the coded image data of each frame 

50 and, for example, it may be stored as a header informa- 
tion of the coded image data of each frame. Alternative- 
ly, the frame feature value may be stored as a separate 
data for image retrieval, independent from the coded im- 
age data. 

55 [0062] Referring to Fig. 4, the image retrieving appa- 
ratus in accordance with the present embodiment in- 
cludes an index information generating unit 401 con- 
nected to an image retrieval executing unit 402, which 
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will be described later, and the image retrieval executing' 
unit 402 connected to index information generating unit 
401. 

[0063] Index information generating unit 401 receives 
as input the frame feature value stored in the image re- s 
trieval information storing apparatus described above, 
extracts a desired featured frame based on the frame 
feature value of each frame, and generates and outputs 
as the index information, the positional information of 
the featured frame. 

[0064] The process executed in the image retrieval 
executing unit 402 will be described later. 
[0065] Referring to Fig. 5, index information generat- 
ing unit 401 includes a scene change frame determining 
unit 501, connected to image retrieval executing unit 
402, a scene change frame index generating unit 502 
connected to scene change frame determining unit 501 , 
a motion featured frame determining unit 503 connected 
to Image retrieval executing unit 402, and a motion fea- 
tured frame index generating unit 504 connected to mo- 
tion featured frame determining unit 503. 
[0066] According to the input frame feature value on 
prediction mode related to coding, a scene change 
frame is determined by scene change frame determin- 
ing unit 501. In the present embodiment, the scene 
change frame is determined by thresholding the frame 
feature value on prediction modes for coding. By chang- 
ing the setting of the threshold value or by adding a proc- 
ess when an adjacent frame is determined to be the 
scene change frame, the scene change frame to be ex- 
tracted varies. 

[0067] In the prior art, the process of extracting a 
scene change frame has been Independent from the im- 
age retrieval process, and the method of determining 
and the like have been fixed. In the present embodi- 
ment, the threshold value for determining the scene 
change or the process of determination is adaptively 
changed tn response to a request for extracting a new 
featured frame output from the image retrieval executing 
unit 402 in the process of image retrieval, as will be de- 
scribed later. Accordingly, the desired scene change 
frames are extracted successively. The scene change 
frame index generating unit 502 generates, as index In- 
formation, positional information of the scene change 
frame determined by the determining unit 501 . 
[0068] Using the frame feature value on motion vector 
as an input, a featured frame related to magnitude of 
motion or the like is determined by motion featured 
frame determining unit 503. In the present embodiment, 
peak position determination is made based on the frame 
feature values on motion vectors of neighboring plurality 
of frames, whereby a frame in which local intensive 
movement is observed is determined. In addition, a 
frame having a magnitude larger than a threshold value 
may be extracted, or a frame of which motion is hardly 
observed and which intensity of motion is on a minimum 
level, may be extracted as the motion featured frame. 
[0069] In the prior art, the process of extracting a mo- 



tion featured frame has been independent from the im- 
age retrieval process, and the method of determination 
and the like have been fixed. In the present embodi- 
ment, the method of determining motion featured frame 
related to what motion feature is of interest is adaptively 
changed in the process of retrieval, in response to a re- 
quest for extracting a new featured frame output from 
image retrieval executing unit 402 in the process of re- 
trieval, which will be described later. Therefore, desired 
motion featured frames are extracted successively. The 
motion featured frame index generating unit 504 gener- 
ates, as index information, positional information of the 
motion featured frame determined by the determining 
unit 503. 

[0070] The scene change frame index information 
and motion featured frame index information obtained 
in this manner are output from index information gener- 
ating unit 401 and input to image retrieval executing unit 
402. 

[0071] Image retrieval executing unit 402 executes 
image retrieval based on the input featured frame index 
information and the coded image data. Image retrieval 
executing unit 402 obtains positional information of the 
featured frame from the featured frame index informa- 
tion, and decodes the corresponding frame, using the 
coded image data. The user recognizes the contents of 
the motion picture by thumbnail display (catalogue dis- 
play) or respective featured frames displayed succes- 
sively, and retrieves a desired image. At this time, when 
it is determined that the featured frame applied as index 
information is not suitable for an image retrieval being 
executed or that the information is insufficient, a request 
for extracting a new featured frame is issued from image 
retrieval executing unit 402 to index information gener- 
ating unit 401 . Receiving the request for extracting a 
new featured frame issued from image retrieval execut- 
ing unit 402, index information generating unit 401 
changes the method of determining featured frames in 
featured frame determining units 501 and 503, deter- 
mines a desired featured frame, and outputs the posi- 
tional information of the featured frame as the index in- 
formation. The output index information is input to image 
retrieval executing unit 402, and image retrieval is newly 
executed. 

[0072] As described with respect to the index informa- 
tion generating unit 401, the featured frame on motion 
pictures, such as the scene change frame and the fea- 
tured frame on motion, may be extracted differently de- 
pendent on the manner of determination, and therefore, 
physical significance of the frame may differ. For exam- 
ple, a frame in which intensive movement is observed 
and a frame in which movement is hardly observed are 
frames of completely different features. 
[0073] When motion picture retrieval is executed, 
whether it is the best to display a frame in which inten- 
sive movement is observed, or a frame with less move- 
ment, or a frame having different features on motion as 
the featured frame of the motion picture depends on the 
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motion picture which is the object of retrieval. Therefore, 
It Is possible that there is a request for switching the fea- 
tured frame adaptlvely in the process of retrieval. In such 
a situation, the image retrieval information storing appa- 
ratus described above can immediately meet the re- s 
quest tor switching the featured frame in the process of 
retrieval, as the image retrieval intormation Is stored in 
the form of frame feature values. 
[0074] Referring to Fig. 6, it is possible to provide an 
image retrieval information storing apparatus which in- 
cludes, in addition to the configuration of the image re- 
trieval information storing apparatus shown in Fig. 1 , an 
index information generating unit 401 generating Index 
information of the featured frames connected to frame 
future value generating unit 102, and an index informa- 
tion storing unit 601 storing index information, connect- 
ed to index information generating unit 401 . In this man- 
ner, index information of predetermined featured frames 
are separately stored together with the frame feature 
values. As the frame feature values and featured frame 
index information are both stored, a request for switch- 
ing a featured frame in the process of retrieval can be 
immediately met by using the frame feature value, and 
high speed retrieval can be executed by using fixed in- 
dex information when there is no request for switching 
in the process of retrieval. 

[0075] Referring to Fig. 7, the image retrieval appara- 
tus corresponding to the image retrieval information 
storing apparatus shown in Fig. 6 includes an index in- 
formation generating unit 401 connected to image re- 
trieval executing unit 402, which will be described later, 
and the image retrieval executing unit 402 connected to 
index information generating unit 401. 
[0076] Index information generating unit 401 has sim- 
ilar structure as index information generating unit 401 
described with reference to Figs. 4 and 5. Therefore, de- 
tailed description thereof is not repeated here. 
[0077] Image retrieval executing unit 402 executes 
image retrieval based on the coded image data and in- 
dex information of featured frames, in the similar man- 
ner as image retrieval executing unit 402 described with 
reference to Fig. 4. At this time, the frame index infor- 
mation is applied not only from the index information 
generating unit 401 but also from prescribed storing lo- 
cations referred by index information storing unit 601 of 
Fig. 6. 

[0078] In the present embodiment, the image retrieval 
information storing apparatus and the image retrieving 
apparatus are different apparatuses between which in- 
dex information and the frame feature values are stored 
and passed. Such information generated by the image 
retrieval information storing apparatus may be directly 
passed to the image retrieving apparatus, and retrieval 
may be executed on-line. 

Second embodiment 

[0079] Referring to Fig. 8, the image retrieval informa- 



tion storing apparatus in accordance with the second 
embodiment includes a coding information reading unit 
101, and a coding information storing unit 801 connect- 
ed to coding information reading unit 101 , for storing the 
coding information read by coding information reading 
unit 101 at a prescribed storing location. Coding Infor- 
mation reading unit 101 is similar to that described in 
the first embodiment. Therefore, detailed description 
thereof is not repeated. 

[0080] The image retrieval information storing appa- 
ratus in accordance with the first embodiment above is 
to store the frame feature value of each frame. In the 
image retrieval information storing apparatus in accord- 
ance with the present embodiment, coding information 
which is the base for extracting the featured frame read 
by coding information reading unit 101 is stored in the 
prescribed storing location by the coding information 
storing unit 801 . By this configuration, what is necessary 
is simply to read the coding information at the prescribed 
location, when the featured frame is to be extracted in 
image retrieval. This improves efficiency as compared 
with successive reading of the original coded image da- 
ta. The location for storing coding information may be a 
position corresponding to the coded image data. For ex- 
ample, it may be stored as header information of the 
coded image data. Alternativety, the coding information 
may be stored independent from coded image data, as 
different data for image retrieval. 

[0081] Referring to Fig. 9, the image retrieving appa- 
ratus in accordance with the second embodiment in- 
cludes a frame feature value generating unit 901 con- 
nected to image retrieval executing unit 902, which will 
be described later, and index information generating unit 
401 connected to frame feature value generating unit 
901 and image retrieval executing unit 902, and the im- 
age retrieval executing unit 902 connected to index in- 
formation generating unit 401. 

[0082] Frame feature value generating unit 901 re- 
ceives coding information stored in image retrieval in- 
formation storing apparatus, and generates and outputs 
the frame feature value in response to a request for ex- 
tracting a frame feature value output from image retriev- 
al executing unit 902. Similar to image retrieval execut- 
ing unit 402 of the first embodiment, image retrieval ex- 
ecuting unit 902 obtains positional information of fea- 
tured frame from index information of the featured 
frames, and decodes the corresponding frame, using 
coded image data. At this time, image retrieval execut- 
ing unit 902 issues a request for extracting a featured 
frame and a request for extracting frame feature value 
to index information generating unit 401 and the frame 
feature value generating unit 901 , respectively. Index in- 
formation generating unit 401 is similar to that described 
in the first embodiment, and therefore detailed descrip- 
tion thereof is not repeated here. 
[0083] In the present embodiment, an image retrieval 
information storing apparatus is implemented in a sim- 
ple configuration in which only the coding information 
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necessary for extracting a featured frame is extracted 
and stored from the originaf coded image data. Thus, 
image retrieval information storing apparatus stores the 
coding information. Therefore, it is possible tor the user 
to change the method of determining a featured frame 
based on the frame feature value, and in addition, to 
change the very method of calculating the frame feature 
value for extracting the featured frame in the process of 
retrieval. Thus, more flexible image retrieval is possible. 
[0084] As in the first embodiment, referring to Fig. 1 0, 
it is possible to provide an image retrieval information 
storing apparatus by adding, to the image retrieval in- 
formation storing apparatus shown in Fig. B, a frame fea- 
ture value generating unit 102 connected to coding in- 
formation reading unit 101 and generating a frame fea- 
ture value for each frame, an index information gener- 
ating unit 401 connected to frame feature value gener- 
ating unit 102 and generating index information of fea- 
tured frame, and an index information storing unit 601 
connected to index information generating unit 401 and 
storing index information. In this manner, index informa- 
tion of a predetermined featured frame is stored sepa- 
rately, together with coding information necessary for 
extracting featured frame. 

[0085] When coding information and featured frame 
index information are both stored, a request for switch- 
ing featured frame in the process of retrieval can be met 
by obtaining the frame feature value based on the stored 
coding information, and in a retrieval without a request 
for switching, retrieval at high speed utilizing fixed index 
information can be performed. 

[0086] Referring to Fig. 1 1 , the image retrieving appa- 
ratus corresponding to the image retrieval information 
storing apparatus shown in Fig. 10 includes a frame fea- 
ture value generating unit 901 connected to Image re- 
trieval executing unit 902, which will be described later, 
an index information generating unit 401 connected to 
frame feature value generating unit 901 and Image re- 
trieval executing unit 902, and the image retrieval exe- 
cuting unit 902 connected to index information generat- 
ing unit 401. 

[0087] Frame feature value generating unit 901 and 
index information generating unit 401 are similar to 
those described with reference to Fig. 9. Therefore, de- 
tailed description thereof is not repeated here. Image 
retrieval executing unit 902 executes image retrieval 
based on the coding image data and index information 
of the featured frame, as image retrieval executing unit 
902 described with reference to Fig. 9. At this time, the 
frame index information is applied not only from the in- 
dex information generating unit 401 but also from a pre- 
scribed storing location referred by index information 
storing unit 601 of Fig. 10. 

[0088] In the image retrieval information storing appa- 
ratuses of the first and second embodiments, the frame 
feature value for each frame and coding information 
necessary for extracting the featured frame are stored. 
These are not incompatible, and therefore the frame 



feature value and the coding information necessary for 
extracting the featured frame may be stored together. 
Further, the index information, the frame feature value 
and the coding information necessary for extracting the 
s featured frame may be all stored and utilized at the time 
of image retrieval. 

Third Embodiment 

[0089] Referring to Fig. 12, the image retrieval Infor- 
mation storing apparatus in accordance with the third 
embodiment includes a coding information reading unit 
1 01 , a frame feature value generating unit 1 02 connect- 
ed to coding information reading unit 101, an index in- 
formation generating unit 401 connected to frame fea- 
ture value generating unit 102, a storing information se- 
lecting unit 1201 connected to coding information read- 
ing unit 101, frame feature value generating unit 1 02 and 
index information generating unit 401 . an index informa- 
tion storing unit 601, a frame feature value storing unit 
1 03 and a coding information storing unit 801 , each con- 
nected to storing information selecting unit 1201. 
[0090] Coding information reading unit 101, frame 
feature value generating unit 102 and index information 
generating unit 401 are similar to those of the first em- 
bodiment. Therefore, detailed description thereof is not 
repeated here. Storing information selecting unit 1201 
selects and outputs one, two or all of index information, 
frame feature value and the coding information neces- 
sary for extracting the featured frame, dependent on the 
situation. Index information storing unit 601 , frame fea- 
ture value storing unit 103 and coding information stor- 
ing unit 801 store the index information, the frame fea- 
ture value and coding information selected by storing 
information selecting unit 1201 at prescribed storing lo- 
cations, respectively. 

[0091] In the first embodiment, it has been described 
that the featured frame for a motion picture such as a 
scene change frame and a featured frame related to mo- 
tion may be extracted differently dependent on the man- 
ner of determination, and that physical meaning of the 
frame may differ. For example, a frame in which rapid 
movement is observed and a frame in which movement 
is very small are frames having different features. When 
motion picture retrieval is executed, whether it is appro- 
priate to display a frame with rapid movement, or a 
frame with less movement, or different frame having dif- 
ferent feature related to motion as the featured frame 
for the motion picture depends on the motion picture 
which is the object of retrieval. Therefore, there may be 
a request for switching the condition of extracting the 
featured frame adaptively in the process of retrieval. 
[0092] In such a situation, the request for switching in 
the process of retrieval may be immediately met when 
the image retrieval information is stored in the form of 
frame feature value or in the form of coding information, 
by the storing information selecting unit 1201. 
[0093] When there is some knowledge of the motion 
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picture which is the object of retrieval in advance and 
appropriate featured frame has been known, it is possi- 
ble by storing the image retrieval information in the form 
of index information of the featured frame by storing in- 
formation selecting unit 1201, to reduce the amount of 
information for image retrieval, and to execute retrieval 
at high speed. Further, when the frame feature value or 
the coding information, and the index information of the 
featured frame are to be stored together by the storing 
information selecting unit 1 201 , it is possible to meet the 
request for switching in the process of retrieval immedi- 
ately, and to execute retrieval at high speed. 
[0094] Referring to Fig. 13, the image retrieving ap- 
paratus in accordance with the third embodiment in- 
cludes a frame feature value generating unit 901 con- 
nected to image retrieval executing unit 902, which will 
be described later, an index information generating unit 
401 connected to frame feature value generating unit 
901 and image retrieval executing unit 902, and the im- 
age retrieval executing unit 902 connected to index in- 
formation generating unit 401. 

[0095] Frame feature value generating unit 901 re- 
ceives, as an input, coding information stored in Image 
retrieval information storing apparatus. Index informa- 
tion generating unit 401 receives, as an input, frame fea- 
ture value generated by frame feature value generating 
unit 901 and the frame feature value stored in the image 
retrieval information storing apparatus. Image retrieval 
executing unit 902 receives as inputs the index informa- 
tion stored in image retrieval information storing appa- 
ratus and the index information generated by index in- 
formation generating unit 401 . Index information gener- 
ating unit 401 is similar to that described in the first em- 
bodiment. Therefore, detailed description thereof is not 
repeated here. Frame feature value generating unit 901 
and image retrieval executing unit 902 are similar to 
those described in the second embodiment. Therefore, 
detailed description thereof is not repeated here. 
[0096] When there is index information, it is possible 
for the Image retrieving apparatus to obtain the position 
of the featured frame based on the index information, 
decode the featured frame based on the input coded im- 
age data, display the same and to execute the image 
retrieval. When a new featured frame is required in the 
process of image retrieval and there is the frame feature 
value, the image retrieving apparatus extracts the new 
featured frame based on the frame feature value, and 
generates the index information to be used for image 
retrieval. 

[0097] When it is detemnined that the frame feature 
value is inappropriate and there is coding Information, 
the image retrieving apparatus generates a new frame 
feature value based on the coding information, extracts 
the new featured frame from thus generated new frame 
feature value, and generates index information to be 
used for Image retrieval. 

[0098] The advantage of storing both the frame fea- 
ture value and the coding information is that image re- 



trieval with such a high degree of freedom is possible. 
[0099] When there is coding Information and not the 
input frame feature value, the request for extracting a 
new featured frame is directly issued to the frame fea- 
s ture value generating unit 901 . 

[0100] In this manner, in the image retrieval informa- 
tion storing apparatus in accordance with the third em- 
bodiment, the information to be stored is selected by the 
storing information selecting unit 1 201 dependent on the 
nature of the image which is the object of retrieval. 
Therefore, appropriate image retrieval information is 
stored, and adaptive image retrieval can be executed 
by the image retrieving apparatus. Further it is possible 
to select the stored information by storing information 
selecting unit 1201 dependent on the rate of execution 
of retrieval or on the restriction of storing capacity. 
[0101] As to the capacity of image retrieval informa- 
tion, the index Information, the frame feature value and 
coding information have capacities larger in this order. 
For example, when the storage capacity of the storing 
location for image retrieval information is small, only the 
index information having small capacity is stored. When 
the storing capacity is sufficiently large and the rate of 
executing retrieval is slow, all the Information is stored 
to ensure degree of freedom of retrieval, and index in- 
formation is fully made use of so as not to lower the ex- 
ecution rate. 

[0102] Further, it is possible to limit the image retrieval 
information to be stored to only two of the index infor- 
mation, frame feature value and the coding information 
necessary for extracting featured frame, and to select 
the stored information by storing information selecting 
unit 1201. 

[0103] Although the present Invention has been de- 
scribed and illustrated in detail, it is clearly understood 
that the same is by way of illustration and example only 
and is not to be taken by way of limitation, the spirit and 
scope of the present invention being limited only by the 
terms of the appended claims. 



Claims 

1. An image retrieval information storing apparatus, 
comprising: 

a coding Information reading unrt (101) reading 
prescribed coding information which will be in- 
formation representing frame feature from cod- 
ed image data; 

a frame feature value generating unit (1 02) con- 
nected to said coding information reading unit 
(101). and generating a frame feature value 
which is a numerical representation ot the 
frame feature, based on said coding informa- 
tion; and 

a frame feature value storing unit (103) con- 
nected to said frame feature value generating 
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unit (102). and storing said frame feature value 
in correspondence with each frame of said im- 
age data. 

2. The image retrieval information storing apparatus s 
according to claim 1 , further comprising: 

an index information generating unit (401 ) con- 
nected to said frame feature value generating 
unit (1 02), determining a featured frame among io 
said image data based on said frame feature 
value, and generating index information which 
is positional information of said featured frame; 
and 

an index information storing unit (601 ) connect- ?£ 
ed to said index information generating unit 
(401) and storing said index information, 

3. The image retrieval information storing apparatus 
according to claim 2, wherein 20 

said coding information includes prediction 
mode Information and motion vector informa- 
tion; and 

said frame feature value generating unit (102) 2S 
includes 

a prediction mode counting unit (201) connect- 
ed to said coding information reading unit (101), 
receiving said prediction mode information and 
counting number of blocks coded in accord- 3o 
ance with respective prediction methods for 
each frame, 

a frequency information converting unit (202) 
connected to said prediction mode counting 
unit (201 ) and outputting prediction mode frame 3S 
feature value which is a numerical representa- 
tion of frame feature, based on the number of 
blocks for each prediction method, 
a motion vector statistic calculating unit (204) 
connected to said coding information reading 40 
unit (1 01 ). and calculating statistics o1 said mo- 
tion vector based on said prediction mode in- 
formation, and 

a statistic information converting unit (205) con- 
nected to said prediction mode counting unit 4S 
(201) and said motion vector statistic calculat- 
ing unit (204), and outputting a motion vector 
frame feature value which is a numerical repre- 
sentation of frame feature, base on outputs of 
said prediction mode counting unit (201) and 50 
said motion vector statistic calculating unit 
(204). 

4. The image retrieving information storing apparatus 
according to claim 1 , wherein said coding informa- 55 
tion includes prediction mode information and mo- 
tion vector information; and 



said frame feature value generating unit (102) 
includes 

a prediction mode counting unit (201) connect- 
ed to said coding information reading unit (101), 
receiving said prediction mode information and 
counting number of blocks coded in accord- 
ance with respective prediction methods for 
each frame, 

a frequency information converting unit (202) 
connected to said prediction mode counting 
unit (201) and outputting a prediction mode 
frame feature value which is a numerical repre- 
sentation of frame feature, based on the 
number of blocks for each prediction method, 
a motion vector statistic calculating unit (204) 
connected to said coding information reading 
unit (101), and calculating said motion vector 
statics based on said prediction mode informa- 
tion, and 

a statistic information converting unit (205) con- 
nected to said prediction mode counting unit 
(201) and said motion vector statistic calculat- 
ing unit (204), and outputting a motion vector 
feature value which is a numerical representa- 
tion of frame feature, based on outputs of said 
prediction mode counting unit (201) and said 
motion vector statistic calculating unit (204). 

5. An image retrieval information storing apparatus, 
comprising: 

a coding information reading unit (101 ) reading 
prescribed coding information which will be in- 
formation representing frame feature from cod- 
ed image data; and 

a coding information storing unit (801 ) connect- 
ed to said coding information reading unit (101 ) 
and storing said coding Information in corre- 
spondence with each frame of said image data. 

6. The image retrieval information storing apparatus 
according to claim 5, further comprising: 

at rame feature value generating unit (1 02) con- 
nected to said coding information reading unit 
(101), and generating a frame feature value 
which is a numerical representation of frame 
feature based on said coding information; 
an index information generating unit (401 ) con- 
nected to said frame feature value generating 
unit (1 02), determining a featured frame among 
said image data based on said frame feature 
value, and generating index information which 
is positional information of said featured frame; 
and 

an index information storing unit (601 ) connect- 
ed to said index information generating unit 
(401) and storing said index information. 
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7. The image retrieval information storing apparatus 
according to claim 6, wherein 

said coding information includes prediction 
mode information and motion vector informa- s 
tion; and 

said frame feature value generating unit (102) 
includes 

a prediction mode counting unit (201) connect- 
ed to said coding information reading unit (101), io 
receiving said prediction mode information and 
counting number of blocks coded in accord- 
ance with respective coding methods for each 
frame, 

a frequency information converting unit (202) is 
connected to said prediction mode counting 
unit (201) and outputting a prediction mode 
frame feature value which is a numerical repre- 
sentation of frame feature based on the number 
of blocks for each prediction method, 20 
a motion vector statistic calculating unit (204) 
connected to said coding information reading 
unit (101) and calculating statistics of said mo- 
tion vector based on said prediction mode in- 
formation, and 25 
a statistic information converting unit (205) con- 
nected to said prediction mode counting unit 
(201) and said motion vector statistic calculat- 
ing unit (204), and outputting a motion vector 
frame feature value which is a numerical repre- 30 
sentation of frame feature based on outputs of 
said prediction mode counting unit (201) and 
said motion vector statistic calculating unit 
(204). 

35 

8. An image retrieval information storing apparatus, 
comprising: 

a coding information reading unit (101) reading 
prescribed coding information which will be in- 40 
formation representing frame feature from cod- 
ed image data; 

a frame feature value generating unit (1 02) con- 
nected to said coding information reading unit 

(101) and generating a frame feature value 4S 
which is a numerical representation of frame 
feature based on said coding information; 

a first storing unit (1201. 103, 801) connected 
to said coding information reading unit (101) 
and to said frame feature value generating unit so 

(102) , and storing said coding information and 
said frame feature value in correspondence 
with each frame of said image data. 

9. The image retrieval information storing apparatus ss 
according to claim 8, further comprising: 

an index information generating unit (401 ) con- 



nected to said frame feature value generating 
unit (1 02), determining a featured frame among 
said image data based on frame feature value 
and generating index information which is po- 
sitional information of said featured frame; 
wherein 

said first storing unit (1201, 103, 801) includes 
a second storing unit (1 201 . 601 , 1 03, 801.) con- 
nected to said coding information reading unit 

(1 01 ) , said frame feature value generating unit 

(102) and said index information generating 
unit (401). and storing said coding information 
and said frame feature value in correspond- 
ence with each frame of said image data and 
said index information. 

10. The image retrieval information storing apparatus 
according to claim 9, wherein 

said second storing unit (1201, 601, 103, 801) 
includes 

a storing information selecting unit (1201 ) con- 
nected to said coding information reading unit 

(101) . said frame feature value generating unit 

(102) and said index information generating 
unit (401), and selecting and outputting any of 
said coding information, said frame feature val- 
ue and said index information. 

an index information storing unit (601 ) connect- 
ed to said storing information selecting unit 
(1201) and storing said index information, 
a frame feature value storing unit (103) con- 
nected to said storing information selecting unit 
(1201) and storing said frame feature value in 
correspondence with each frame of said image 
data, and 

a coding information storing unit (801 ) connect- 
ed to said storing information selecting unit 
(1201) and storing said coding information in 
correspondence with each frame of said image 
data. 

11. The image retrieval information storing apparatus 
according to claim 8, wherein 

said first storing unit(1201, 103, 801) includes 
a storing information selecting unit(1201) con- 
nected to said coding information reading unit 
(101) and said frame feature value generating 
unit(102), and selecting and outputting any of 
said coding information and said frame feature 
value, 

a frame feature value storing unit(103) con- 
nected said storing information selecting unit 
(1201) and storing said frame feature value in 
correspondence with each frame of said image 
data, and 

a coding information storing unit(801) connect- 
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ed said storing information selecting unft(1 201 ) 
and storing said coding information in corre- 
spondence witli each tranne of said image data. 

12. Tine image retrieval information storing apparatus 5 
according to claim 10, wherein 

said coding information includes prediction 
mode information and motion vector informa- 
tion; and 10 
said frame feature value generating unit (102) 
includes 

a prediction mode counting unit (201) connect- 
ed to said coding information reading unit (101), 
receiving said prediction mode information and '5 
counting number of blocks coded in accord- 
ance with respective prediction methods for 
each frame. 

a frequency information converting unit (202) 
connected to said prediction mode counting 20 
unit (201) and outputting a prediction mode 
frame feature value which is a numerical repre- 
sentation of frame feature based on the number 
of the blocks for each prediction method, 
a motion vector statistic calculating unit (204) 2S 
connected to said coding information reading 
unit (1 01 ), and calculating statistics of said mo- 
tion vector based on prediction mode informa- 
tion, and 

a statistic information converting unit (205) con- 3o 
nected to said prediction mode counting unit 
(201 ) and said motion vector statistic calculat- 
ing unit (204), and outputting a motion vector 
frame feature value which is a numerical repre- 
sentation of frame feature, based on outputs of 35 
said prediction mode counting unit (201 ) and 
said motion vector statistic calculating unit 
(204). 

13. The image retrieval information storing apparatus 40 
according to frame 9. wherein 

said coding information includes prediction 
mode information and motion vector informa- 
tion; and 45 
said frame feature value generating unit (102) 
includes 

a prediction mode counting unit (201) connect- 
ed to said coding information reading unit (1 01 ), 
receiving said prediction mode information and so 
counting number of blocks coded in accord- 
ance with respective prediction methods for 
each frame, 

a frequency information converting unit (202) 
connected to said prediction mode counting 55 
unit (201) and outputting a prediction mode 
frame feature value which is a numerical repre- 
sentation of frame feature based on the number 



of the blocks for each prediction method, 
a motion vector statistic calculating unit (204) 
connected to said coding information reading 
unit (101), and calculating statistics of said mo- 
tion vector based on prediction mode informa- 
tion, and 

a statistic information converting unit (205) con- 
nected to said prediction mode counting unit 
(201) and said motion vector statistic calculat- 
ing unit (204), and outputting a motion vector 
frame feature value which is a numerical repre- 
sentation of frame feature, based on outputs of 
said prediction mode counting unit (201) and 
said motion vector statistic calculating unit 
(204). 

14. The image retrieval information storing apparatus 
according to claim 8, wherein 

said coding information includes prediction 
mode information and motion vector informa- 
tion; and 

said frame feature value generating unit (102) 
includes 

a prediction mode counting unit (201) connect- 
ed to said coding information reading unit (1 01 ), 
receiving said prediction mode information and 
counting number of blocks coded in accord- 
ance with respective prediction methods for 
each frame, 

a frequency information converting unit (202) 
connected to said prediction mode counting 
unit (201) and outputting a prediction mode 
frame feature value which is a numerical repre- 
sentation of frame feature based on the number 
of the blocks for each prediction method, 
a motion vector statistic calculating unit (204) 
connected to said coding information reading 
unit (101), and calculating statistics of said mo- 
tion vector based on prediction mode informa- 
tion, and 

a statistic information converting unit (205) con- 
nected to said prediction mode counting unit 
(201) and said motion vector statistic calculat- 
ing unit (204), and outputting a motion vector 
frame feature value which is a numerical repre- 
sentation of frame feature, based on outputs of 
said prediction mode counting unit (201) and 
said motion vector statistic calculating unit 
(204). 

15. An image retrieving apparatus, comprising; 

an index information generating unit (401) re- 
ceiving a frame feature value which is a numer- 
ical representation of frame feature corre- 
sponding to each frame of coded image data, 
determining a featured frame among said im- 
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age data based on said frame feature value in 
accordance with a request for extracting a fea- 
tured frame, and generating index information 
which is positional information of said featured 
frame; and 5 
a first image retrieval executing unit (402) con- 
nected to said index information generating unit 
(401 ), transmitting said request for extracting a 
featured frame to said index information gener- 
ating unit (401), receiving said index informa- io 
tion f rom said index information generating unit 
(401), receiving said image data from outside, 
and outputting a frame specified based on said 
index information. 

IS 

1 6. The image retrieving apparatus to claim 1 5, wherein 

said first image retrieval executing unit (402) 
includes a second image retrieval executing unit 
(402) connected to said index information generat- 
ing unit (401), transmitting said request for extract- 20 
ing a featured frame to said index information gen- 
erating unit (401), receiving said index information 
from said index information generating unit (401 ), 
receiving said image data and index information 
from outside, and outputting a frame specified 2S 
based on said index information received from said 
index information generating unit (401) and from 
outside. 

17. The image retrieving apparatus according to claim 30 
16, wherein 

said index information generating unit (401) in- 
cludes 

a frame determining unit (501 , 503) receiving 35 
said frame feature value and said request for 
extracting a featured frame, comparing said 
frame feature value and a threshold value in ac- 
cordance with said request for extracting a fea- 
tured frame, and determining said featured 40 
frame, and 

an index generating unit (502, 504) connected 
to said frame determining unit and generating 
index information which is positional informa- 
tion of said featured frame. 4S 

18. The image retrieving apparatus according to claim 
15, wherein said index information generating unit 
(401 ) includes 

so 

a frame determining unit (501 , 503) receiving 
said frame feature value and said request for 
extracting a featured frame, comparing said 
frame feature value and a threshold value in ac- 
cordance with said request for extracting a fea- 55 
tured frame, and determining said featured 
frame, and 

an index generating unit (502, 504) connected 



to said frame determining unit and generating 
index information which is positional informa- 
tion of said featured frame. 

19. An image retrieving apparatus, comprising: 

a frame feature value generating unit (901) re- 
ceiving prescribed coding information included 
in coded image data which wilt be information 
representing frame feature, and generating a 
frame feature value which is a numerical repre- 
sentation of the frame feature based on said 
coding information ; 

an index information generating unit (401 ) con- 
nected to said frame feature value generating 
unit (901), receiving said frame feature value 
from said frame feature value generating unit 
(901), determining a featured frame among 
said image data based on said frame feature 
value in accordance with a request for extract- 
ing a featured frame, and generating index in- 
formation which is positional information of said 
featured frame, and 

a first image retrieval executing unit (902) con- 
nected to said frame feature value generating 
unit (901) and said index information generat- 
ing unit (401), transmitting said request for ex- 
tracting a frame feature value to said frame fea- 
ture value generating unit (901), transmitting 
said request for extracting a featured frame to 
said index information generating unit (401 ), re- 
ceiving said index information from said Index 
information generating unit (401) and output- 
ting a frame specified based on said index in- 
formation. 

20. The image retrieving apparatus according to claim 

19, wherein 

said first image retrieval executing unit (902) 
includes a second image retrieval executing unit 
(902) connected to said frame feature value gener- 
ating unit (901 ) and said index information generat- 
ing unit (401), transmitting said request for extract- 
ing a frame feature value to said frame feature value 
generating unit (102), transmitting said request for 
extracting a featured frame to said index informa- 
tion generating unit (401), receiving said Index in- 
formation from said index information generating 
unit (401) and from outside, receiving said image 
data from outside, and outputting a frame specified 
based on said Index Information. 

21. The image retrieving apparatus according to claim 

20, wherein 

said index information generating unit (401 ) in- 
cludes 

a frame determining unit (501 , 503) receiving 
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said frame feature value and said request for 
extracting .a featured frame, comparing said 
frame feature value and a threshold value in ac- 
cordance with said request for extracting a fea- 
tured frame, and determining said featured 
frame, and 

an index generating unit (502, 504) connected 
to said frame determining unit and generating 
index information which is positional informa- 
tion of said featured frame. 

22. The image retrieving apparatus according to claim 
19, wherein 

said index information generating unit (401) In- 
cludes 

a frame determining unit (501. 503) receiving 
said frame feature value and said request for 
extracting a featured frame, comparing said 
frame feature value and a threshold value in ac- 
cordance with said request for extracting a fea- 
tured frame, and determining said featured 
frame, and 

an index generating unit (502. 504) connected 
to said frame determining unit and generating 
index information which is positional informa- 
tion of said featured frame. 

23. An image retrieving apparatus, comprising: 

a frame feature value generating unit (901) re- 
ceiving prescribed coding information included 
in coded image data, which will be information 
representing frame feature, and generating a 
frame feature value which is a numerical repre- 
sentation of the frame feature based on said 
coding information, in accordance with a re- 
quest for extracting a frame feature value; 
an index information generating unit (401) con- 
nected to said frame feature value generating 
unit (901), receiving said frame feature value 
from said frame feature value generating unit 
(901 ) and from outside, determining a featured 
frame among said image data based on said 
frame feature value in accordance with a re- 
quest for extracting a featured frame, and gen- 
erating index information which is positional in- 
formation of said featured frame; and 
a first image retrieval executing unit (902) con- 
nected to said frame feature value generating 
unit (901) and said index information generat- 
ing unit (401), transmitting said request for ex- 
tracting a frame feature value to said frame fea- 
ture value generating unit (901), transmitting 
said request for extracting a featured frame to 
said index information generating unit (401 ), re- 
ceiving said index information from said infor- 
mation generating unit (401), receiving said im- 



age data from outside, and outputting a frame 
specified by said index information. 

24. The image retrieving apparatus according to claim 

5 23, wherein 

said first image retrieval executing unit (902) 
includes a second image retrieval executing unit 
(902) connected to said frame feature value gener- 
ating unit (901 ) and said index information generat- 

TO ing unit (401), transmitting said request for extract- 
ing a frame feature value to said frame feature value 
generating unrt (901), transmitting said request for 
extracting a featured frame to said index informa- 
tion generating unit (401), receiving said index in- 

'5 formation from said index information generating 
unit (401) and from outside, receiving said image 
data from outside, and outputting a frame specified 
based on said index information. 

20 25. The image retrieving apparatus according to claim 
24, wherein 

said index information generating unit (401) in- 
cludes 

2S a frame determining unit (501, 503) receiving 

said frame feature value and said request for 
extracting a featured frame, comparing said 
frame feature value and a threshold value in ac- 
cordance with said request for extracting a fea- 

•30 tured frame, and determining said featured 

frame, and 

an index generating unit (502, 504) connected 
to said frame determining unit and generating 
index information which is positional informe- 
rs tion of said featured frame. 

26. The image retrieving apparatus according to claim 
23, wherein 

40 said index information generating unit (401 ) in- 

cludes 

a frame determining unit (501, 503) receiving 
said frame feature value and said request for 
extracting a featured frame, comparing said 
^ frame feature value and a threshold value in ac- 

cordance with said request for extracting a fea- 
tured frame, and determining said featured 
frame, and 

an index generating unit (502, 504) connected 
50 to said frame determining unit and generating 

index information which is positional informa- 
tion of said featured frame. 
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